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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
Applicant : J.B. MOORE et al. 

Appl. No. : Not Yet Assigned (National Stage of PCT/AU00/00022) 

Filed : Concurrently Herewith (International Filing Date January 14, 2000) 

For : RESOLUTION INVARIANT PANORAMIC IMAGING 

PRELIMINARY AMENDMENT 

Assistant Commissioner of Patents 
Washington, D.C. 20231 

Sir: 

Prior to calculation of the fees and an examination of the above-identified patent 
application, the Examiner is respectfully requested to amend the claims, as follows: 
TN THE CLAIMS 

Please amend the claims, as follows (a marked-up copy of the claims is attached 
hereto): 

3. (Amended-Clean Text) A panoramic imaging system as claimed in claim 1 
wherein the profile of the or each convex portion at least approximates a profile defined 
in polar co-ordinates by the equation: 

—=rcot[ -1/2/(1 +a(0y9] 
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where r is the radial distance from the reflective surface to the imaging device 
0 is the angle from the optical axis of the imaging device 
a(0) is the mirror gain given by 

a(0)=5 a [tan(0)+tan 3 (0)] 

2($-jfc) 
a ~tan 2 (0)-tan 2 (0) 

<j) and 4> are the maximum and minimum elevations viewed 0 and 0 are the maximum and 
minimum radial angles imaged. 

6. (Amended-Clean Text) A panoramic imaging system as claimed in claim 1 
including a first reflector surface having at least two of said convex portions arranged to 
respectively provide at least partially overlapping panoramic second fields of view for 
range determination. 

9. (Amended-Clean Text) A panoramic imaging system as claimed in claim 1 
including two of said first reflective surfaces each having an associated image plane with 
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corresponding first fields of view, and at least one convex portion of each first reflective 
surface providing respective panoramic second fields of view, said first reflective surface 
being arranged back to back such that said reflective second fields of view at least 
partially overlap. 

10. (Amended-Clean Text) A panoramic imaging system as claimed in claim 1 
further including a second surface interposed between the image plane and said second 
reflective surface. 

13. (Amended-Clean Text) A panoramic imaging system as claimed in claim 10 
wherein said second reflective surface is substantially planar. 

16. (Amended-Clean Text) A reflective surface as claimed in claim 14 wherein 
the profile of the or each convex portion at least approximates a profile defined in polar 
co-ordinates by the equation: 

—=rcot[ -1/2 J(l +a(Q)dQ] 
dQ 

where r is the radial distance from the reflective surface to the imaging device 
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0 is the angle from the optical axis of the imaging device 
a(0) is the mirror gain given by 

a(0)=B K [tan(0)+tan 3 (6)] 

B 2($-&) 
° tan 2 (0)-tan 2 (0) 

(j) and (J) are the maximum and minimum elevations viewed 0 and 0 are the maximum and 
minimum radial angles imaged. 

19. (Amended-Clean Text) A reflective surface as claimed in claim 14 including 
a first reflector surface, having at least two of said convex portions arranged to 
respectively provide at least partially overlapping panoramic second fields of view for 
range determination. 

REMARKS 

The Examiner is respectfully requested to enter the foregoing amendment to delete 
multiple dependency prior to calculation of the filing fees and examination of the above- 
identified patent application. 
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The amendments to the claims made in this amendment have not been made to 
overcome the prior art, and thus, should be considered to have been made for a purpose 
unrelated to patentability, and no estoppel should be deemed to attach thereto. 

Should there be any questions, the Examiner is invited to contact the undersigned 
at the below-listed telephone number. 



July 9, 2001 

GREENBLUM & BERNSTEIN, P.L.C. 
1941 Roland Clarke Place 
Reston,VA 20191 
(703)716-1191 



Respectfully submitted, 
J.B. MOORE et al. 




Reg. No. 29,027 
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MARKED-UP COPY OF CLAIMS 

3. (Amended) A panoramic imaging system as claimed in claim 1 [or claim 2] 
wherein the profile of the or each convex portion at least approximates a profile defined 
in polar co-ordinates by the equation: 

—=rcot[-mf(l +a(0)tf0] 

where r is the radial distance from the reflective surface to the imaging device 
0 is the angle from the optical axis of the imaging device 
a(0) is the mirror gain given by 

a(0)=fl a [tan(0)+tan 3 (0)] 



b 2(<j>-& 
° tan 2 (0)-tan 2 (6) 



(J) and ^ are the maximum and minimum elevations viewed 0 and 0 are the maximum and 
minimum radial angles imaged. 
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6. (Amended) A panoramic imaging system as claimed [in any one of claims 1 to 
5] in claim 1 including a first reflector surface having at least two of said convex portions 
arranged to respectively provide at least partially overlapping panoramic second fields of 
view for range determination. 

9. (Amended) A panoramic imaging system as claimed [in any one of claims 1 to 
8] in claim 1 including two of said first reflective surfaces each having an associated 
image plane with corresponding first fields of view, and at least one convex portion of 
each first reflective surface providing respective panoramic second fields of view, said 
first reflective surface being arranged back to back such that said reflective second fields 
of view at least partially overlap. 

10. (Amended) A panoramic imaging system as claimed [in any one of claims 1 
to 9] in claim 1 further including a second surface interposed between the image plane 
and said second reflective surface. 

13. (Amended) A panoramic imaging system as claimed in [any of claims 10 to 
12] claim 10 wherein said second reflective surface is substantially planar. 
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16. (Amended) A reflective surface as claimed in claim 14 [or claim 18] wherein 
the profile of the or each convex portion at least approximates a profile defined in polar 
co-ordinates by the equation: 

— =rcot[-V2kl+a(Q)dd] 



where r is the radial distance from the reflective surface to the imaging device 
0 is the angle from the optical axis of the imaging device 
a(0) is the mirror gain given by 



a(6)=B c [tan(6)+tan 3 (6)] 

2($-&) 
a tan 2 (0)-tan 2 (0) 

<]) and 4> are the maximum and minimum elevations viewed 0 and 0 are the maximum and 
minimum radial angles imaged. 
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19. (Amended) A reflective surface as claimed [in any one of claims 14 to 18] in 
claim 14 including a first reflector surface, having at least two of said convex portions 
arranged to respectively provide at least partially overlapping panoramic second fields of 
view for range determination. 
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RESOLUTION INVARIANT PANORAMIC IMAGING 



FIELD OF THE INVENTION 

This invention relates to generating wide angle images of spaces, generally referred to as 
5 panoramic imaging. 

BACKGROUND ART 

Panoramic imaging is becoming an important tool in the area of mobile robotics and machine 
10 vision. There are many documented methods for recording a panoramic view of a scene. One 
simple method involves having a series of cameras mounted on a ring to give views around 
the entire 360° of horizon. This involves, say, four cameras if they each have a field of view 
of 90° and some integration of images. There are also a number of single camera methods for 
panoramic imaging, including rotating a camera about its vertical axis and taking pictures 
15 continuously to obtain a full panoramic view. Another approach uses wide angle lenses to 
achieve a large field of view, but these lenses are heavy, expensive and distort the image. 

An attractive approach to panoramic imaging is to mount a single fixed camera under a curved 
reflective surface covering a hemisphere such as with a conical, spherical, hyperboloidal, or 

20 other profile. The optical axis of the camera is aligned with the central axis of the mirror. A 
known family of constant gain reflective surfaces have the advantage that they can produce 
large fields of view such as for a hemispherical or hyperboloidal mirror yet preserve a linear 
relationship between changes in angles of incidence and reflection of light rays viewed by the 
camera. This linear relationship simplifies image processing and ensures constant elevational 

25 resolution of the image. The shape of the surface is determined by the gain of the linear 
relationship. For a unity gain, the surface is a cone; for higher gains, the surface is specified 
by a family of polynomial functions. For ease of explanation in this specification the 
panoramic plane will be considered as being horizontal and the field of view as vertical as 
would be the case for a robot moving in a horizontal plane. It will be apparent that in the 

30 general case orientation of the planes is arbitrary. 
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A1I the mirror shapes mentioned above share a common draw back. That is that the CCD 
cameras used for imaging invariably have uniform Cartesian arrays of pixels to capture the 
polar image of the scene, and so the pixel density per solid angle increases with the radius of 
the polar image. The unwarping process transforms the image from polar to Cartesian 
5 coordinates so that the angular coordinate in the original polar image maps to the x-coordinate 
in the unwarped image while the radial coordinate maps to the y-coordinate. Thus the pixel 
density in the unwarped image varies from low for small x values which correspond to the 
centre of the original image to high for large x values which correspond to the outer rim of 
the polar image. This is illustrated in Figure 1 which shows the unwarping of an image 
1 0 captured with a hyperboloidal mirror. The variation in image quality is clearly evident in the 
unwarped version. 

One way to circumvent this problem is to use a specially designed CCD camera with a polar 
array of pixels with a pixel density which decreases with radius. There are alignment 
1 5 problems with such an approach. 

DISCLOSURE OF THE INVENTION 

In a first aspect this invention provides a panoramic imaging system including an imaging 
device having an image plane and a first field of view, a first reflective surface having at least 
20 one circularly symmetric portion convex in a radial direction disposed in said first field of 
view to provide an expanded panoramic second field of view, the profile of the or each 
convex portion providing a varying gain between the fields of view in the radial direction to 
limit variation in the solid angle of view across the image plane of the imaging device. 

25 Preferably, the profile of the convex portion provides a substantially uniform solid angle of 
view across the image plane. That is, the shape ensures that the resolution in the image is 
invariant to changes in elevation. Thus, where the imaging system involves a device with an 
array of uniformly spaced pixels in the image plane, the shape of the reflective surfaces results 
in solid angle pixel density invariance. 

30 
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The profile of the reflective surface in polar coordinates is preferably determined by solving 
the equation 

=rcot[-'/2 J(l + a(Q))do\ 

where r is the radial distance from the reflective surface to the imaging device 
5 0 is the angle from the optical axis of the imaging device 

a (0) is the mirror gain given by 

a (0) = B c [tan (6) + tan 3 (0)] 

B 2($ - ft 

° tan 2 (0) - tan 2 (fi) 

10 

$ and 4> are the maximum and rninimum elevations viewed B and 0 are the maximum and 
minimum radial angles imaged. 

In one approach r can be plotted against 0 at selected intervals to describe the profile by 
1 5 solving the above equation for selected values of 0. For example detenriining values of r for 
incremental values of 0 of about 1/5° has been found to produce a sufficiently accurate profile 
for practical application. 

There are a number of methods for panoramic range finding. One method uses a cone mirror 
20 above a camera. The camera mirror assembly is either displaced during image collection, or 
two camera mirror assemblies are used to obtain the two views necessary for range finding. 
Although this method provides range information in the horizontal plane at video rates, its 
drawbacks are that no range information is available in the vertical (elevation) direction, 
objects must be more than a minimum distance from the camera and there may be a blind spot 
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due to the second camera system. 

A discontinuous, axially symmetric mirror, which is in essence a coaxial mirror pair, mounted 
above a camera to obtain two views of a panoramic scene for stereo disparity range finding 
5 is known. There are however no proposals concerning specific mirror shapes to achieve 
specific desirable properties. Additionally, known constant gain mirror profiles have been 
generalised to derive a family of such coaxial mirror pair profiles for panoramic stereo 
imaging and processing based on disparities in the vertical plane. 

10 In another aspect this invention provides for range finding using a panoramic imaging system 
containing two resolution invariant mirrors. Preferably the mirror or reflector surface has at 
least two of said convex portions arranged to respectively provide at least partially 
overlapping panoramic second fields of view for range determination. The second fields of 
view are preferably substantially co-incident. In the preferred form of the invention the two 

15 convex portions form a continuous mirror or reflective surface. 

In a further aspect this invention provides a design for a back to back stereo mirror system 
with the desirable property of equal pixel sharing between two cameras and thus the two 
stereo images. The stereo cone in this case is preferably symmetric in the directions 

20 orthogonal to the camera axis which is a desirable property for some applications. In this 
aspect of the invention the imaging system preferably includes two first reflective surfaces 
each having an associated image plane with corresponding first fields of view, and at least one 
convex portion of each first reflective surface providing respective panoramic second fields 
of view, said first reflective surface being arranged back to back such that said reflective 

25 second fields of view at least partially overlap. 

A second reflective surface can, in some applications be interposed between the image plane 
and the second reflective surface. This allows positioning of the imaging device for example 
behind the first reflective surface. In some variations an apperture can be provided in the first 
30 reflective surface to provide the first field of view from the imaging device. 
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In another aspect this invention provides a reflective surface for use in a panoramic imaging 
system including an imaging device having an imaging plane and a first field of view, said 
reflective surface having at least one circularly symmetric portion convex in a radial direction 
with a profile providing varying gain in the radial direction between an expanded panoramic 
5 second field of view provided by the reflective surface and the first field of view to limit 
variation in the solid angle of view across the image plane of the imaging device. 

In yet a further aspect this invention provides mirrors having minimal intrusive designs, which 
intrude to a minimal extent into the viewing "hemisphere". These are also termed forward 
10 facing designs. They involve an additional planar mirror and camera relocation within the 
primary reflective surface. The attraction of this arrangement is that the first reflective mirror 
surface profile is the same design as in a more conventional arrangement. 

The invention will be further described, by way of example only with reference to the 
15 accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates an un warping process for a prior art panoramic imaging system; 
Figure 2 schematically shows the relationship between camera image and horizontal 
20 view direction in a panoramic imaging system; 
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Figure 3 illustrates geometric relationships between a reflecting surface and a camera 
used to derive mirror profiles according to this invention; 

Figures 4A and 4B, are graphs showing a comparison of a constant gain mirror with 
5 a variable gain mirror used in the imaging system according to this invention; 

Figures 5A and 5B, shows ray traced scenes respectively reflected in constant and 
variable gain mirrors; 

Figures 6A and 6B, graphically illustrates a comparison of panoramic imaging systems 
respectively utilising double constant and variable gain mirror configurations; 
10 Figures 7A and 7B, shows raced traced images of scenes respectively corresponding 

to panoramic imaging systems utilising double constant and variable gain mirror 
configurations; 

Figure 8 schematically illustrates relationships between camera and reflective surfaces 
used in range calculation utilising a resolution invariant double mirror according to this 
15 invention; 

Figure 9 schematically illustrates a back to back mirror configuration according to this 
invention; 

Figure 10 schematically illustrates a double back to back mirror configuration 
according to this invention; 
20 Figure 11 schematically illustrates a forward looking panoramic imaging system 

according to this invention; and 

Figure 12 shows a system utilising a combination of the arrangements in Figures 10 

and 11. 

25 BEST MODE FOR CARRYING OUT THE INVENTION 

The various aspects of this invention will, for clarity, be described under separate 
subheadings. 

1 Resolution Invariant Mirror Families 

30 
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This section describes a family of mirror designs that achieve the objective of resolution 
invariance, or equivalently solid angle pixel density invariance. 

5 1.1 Constant Image Pixel Density - The Variable Gain (a) Mirror 

In accordance with one aspect of this invention resolution invariance is achieved by adjusting 
the mirror profile to image relatively less of the scene in the centre of the image and relatively 
more at the perimeter. That is, a mirror profile is selected to maintain a constant relationship 
10 between the pixel density and the angle of elevation in the scene or more precisely, the solid 
angle. The rnirror gain a, is the relationship between the change in elevation of rays incident 
on the mirror and the change in the angle of rays reflected into the camera as follows 



(1) 



15 where S&is the change in vertical elevation and 60 is the change in angle of reflected rays 
received by the camera. With resolution invariance a becomes a function of image angle 6 
which is related to the radial coordinate in the image, p, as shown in Figure 2. 

Figure 3 schematically shows an imaging system including an imaging device in the form of 
20 a camera having an image plane and a first field of view. A reflective surface or mirror is . . . 
in the first field of view to provide an expanded panoramic second field of view. The surface 
is circularly symmetric and convex in a radial direction. 

Consider a mirror profile (r,Q) in polar coordinates where r is the radial distance to the 
25 camera and 8 is the angle from the optical axis of the camera to the point on the mirror 
surface as shown in Fig. 3. The angle of incidence of a light ray relative to the mirror is y 
and the angle of an incoming light ray with respect to the vertical is Then 
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V - tan ^ _ J (2) 

subject to the geometric constraint (from the law of reflection) 

2y + 6 + $ = % (3) 



5 Differentiating (2) and (3) with respect to 8 




From (2) 



From (3) 



so, substituting a from (1) gives 




(4) 



10 

Now, for a variable gain mirror, a is a function of image angle 0 (related to the radial 
coordinate in the image, p) so (4) becomes 

wM?)]-*-<™ < s > 



or, rearranging 

15 
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~=r cotp/2/O + a(6))</0] 
dd 



(6) 



The equation for the mirror gain, a (0) to achieve pixel density invariance can be found using 
the following theory. 

5 

1.1.1 Pixel Density Invariance Profiles 

There are p(p) pixels in an area of radius p in the image. More formally, there are 

/7(p) = 7lKp 2 

10 pixels in an area of radius p , where k is the number of pixels per unit area, a constant. 
Differentiating by p gives 



Now, the radius in the image, p is related to the radial angle of a ray reflected from the 
15 mirror, 0 by the focal length of the camera,/ (a constant) 

p =/tan(0) (8) 
so differentiating p(p) by 0 and substituting (7) and (8) gives 



20 
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fo(p) _ dpi?) dp 

dQ dp ae 

-2nK P f dtan ^ (9) 

VJ ae 

= 27rK/ 2 tan(0)(l + tan 2 (0)) 



Now, it is required that the image pixel density be invariant to angle of elevation in the scene 
which leads to more of the scene being imaged towards the perimeter, so 

5 

p(p) = P4> + C(<|>) (10) 



where (5 and C(<b) are constants. Differentiating both sides of (10) by 4> and substituting (1) 
and (9) gives 



3 §± = ap(p) 

= 2mc/ 2 tan(0)(l + tan 2 (0)) 
«(6) 



10 

Rearranging (11) gives 



a(0) = ^ tan(6)[l + tan 2 (0)] 

= B a [tan(0) + tan 3 (0)] 



(12) 



where B c is a constant. Integrating this expression for a(0) by 0 gives an expression for <J> 
15 (see (1», the elevation of an object imaged at angle 0. That is 
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= — tan 2 (0) + (j>(6 = 0) 



(13) 



where (b (0 = 0) is a constant of integration. 

The constants B a and $(0 = 0) can be determined from the maximum and minimum values 
5 of 0 and (j> which are known for a desired mirror configuration, using (13). 



It appears not possible to find an analytical solution to (6) if a is a function of 0, so there is 
no explicit equation for the mirror shape. Instead, a differential equation solver is needed to 
10 find solutions to (6) over the range of 0 (the mirror surface). 

Figures 4A and 4B show for comparison a constant gain mirror and a variable gain mirror 
with the same camera field of view and range of elevations imaged. The rays shown are 
constantly spaced in 0, with about 2° between each ray. It is clear from Fig. 4A that in the 

15 constant gain case these rays are constantly spaced in 4>, with about 8.5° between each ray, 
and from Fig. 4, that the spacing between the rays in the variable gain case increases with 
increasing cf>. So, in the variable gain case, a greater proportion of the scene is imaged 
towards the outer edge of the polar image. This is also shown in Figures 5 A and 5B, ray 
traced images reflected in a constant gain and variable gain mirror with the same range of 

20 elevations visible. 



2(cj> - 40 



a tan 2 (0) - tan 2 (6) 
<j>(e = 0) = £ - ^tan 2 (0) 



(14) 
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1.2 Panoramic Stereo Using a Variable Gain Mirror 

A mirror with two convex portions or a double mirror is required. The radial profile of a 
5 double mirror is shown in Figure 8. The mirror arrangement for panoramic stereo with 
variable gain mirrors will necessarily be different than for constant gain mirrors due to the 
variation of the mirror gain, a. The gain must vary in a constant fashion over the entire 
double mirror so that the constant pixel density 

theorem will hold over the entire image. If the minimum and maximum elevations viewed ($> 
1 0 and 7p ) are to be equal for both mirrors in the double mirror system, the range of reflected 
angles {Jd - 6 ) cannot be equal for the two mirrors. The minimum and maximum angles of 
reflected rays captured by the camera over the entire mirror surface are known from camera 
geometry. Therefore the minimum ray reflected from the lower mirror {6^ ) and the 
maximum ray reflected from the upper mirror (6 2 ) are known. So, since (12) holds over the 
15 entire mirror, B a is constant, and from (14) 

2$ - j>) _ 2($ - jfe) 
(tan 2 (Wj - tan 2 ^)) tan 2 (W 2 ) - tan 2 (0,) 

tan 2 (0^) - tan 2 ^) = tan 2 ((^) - tan 2 (a,) (15) 

tan 2 (6^) + tan 2 (0,) = tan 2 (0j) + tan 2 ^) 



It is desirable to minimise the gap in the radial direction between the images from the two 
mirrors so as to maximise usage of the camera field of view. For minimum gap = 6^ , 



2 tan 2 (6,) = tan 2 (0 2 ) + tan 2 ^) 

' tan 2 (0^) + tan 2 ^) 



(16) 



20 
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Figures 6A, 6B and 7A, 7B show graphical and ray traced comparisons of constant and 
variable gain double mirror systems viewing the same scene. 

2.4 Calculation of Range for a Variable Gain Panoramic Stereo System 

5 

The information available for range calculation are the image angles for a single object 
reflected in both mirrors, 6, and 0 2 as shown in Figure 8. The two mirrors 6j and 8 2 form 
a reflective surface. The differential equations (6) for the surfaces are known. In the 
calculations that follow only the lower mirror is examined as the results are identical for the 
10 upper mirror. 

In order to fmd the position of object P, the equations of the incident beams from P to each 
mirror reflection point (r,, 0,) and (r 2 , Bj) must be found. These equations can then be solved 
simultaneously to give the position of object P, (x P , y P ). 







1 -m n 


-i 








1 -m I2 







m n -m n m I2 -m n 

1 1_ 

m u -m n m I2 -m n 



(17) 
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where m n is the gradient of the incident beam to the lower mirror and C n is the equation 
constant. The equation constant is given by 

c u=yr m u x i (is) 



where 

5 

Xj = rj sin 6j 

y } = r x cos 0 1 (19) 

are the Cartesian coordinates of the reflection point (r lt 8j). The gradient of the incident 
10 beam is found using the law of reflection 



(20) 



where m R1 is the gradient of the reflected beam from the lower mirror to the camera and 
1 5 dy l /dx l is the gradient of the lower mirror profile at the reflection point. The gradient of the 
reflected beam is 



= tand. 



(21) 



The gradient of the mirror profile for the lower variable gain mirror is found as in the 
constant gain case, from 

20 
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dy x _ dy ] 
dr 

~~^\ 
~dQ 



cos6, -rjsinOj 



(22) 



smy, +r, cost! 



where dr/d6 for either mirror of the variable gain mirror configuration is found by integrating 
(5) and substituting (12). 



(Q))a 



~{ r it) = ~¥ " h J* (tan(e) + tan3(6)) * (23) 



-1q - ^tan 2 (0) + £> 
2 4 



where Z> is a constant of integration. Rearranging (23) gives 



dr J 
— = r cot 



-0 - — tan 2 (6) + D 
2 4 



(24) 



Now from (23) and (2), 



D = y + —6 + — tan 2 (6) 
r 2 4 ' 



(25) 



10 so, for the lower variable gain mirror profile 
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similarly for D 2 , for the upper variable gain profile. 

So, by substituting (24) into (22) gives the gradient of the variable gain mirror profiles at any 
5 point. Note that as in the constant gain case, the gradient depends only on 6. 

The equation constants for the incident beam equations from (18) require the polar coordinates 
of the^reflection points from each mirror, (r„ 6 } ) and (r 2 , . Since the variable gain mirror 
equations are not known exactly, r, and r 2 must be found using a differential equation solver 
10 to find solutions to (26) at and 0 2 . 

2 Back-to-back Stereo Mirror Families 

A key disadvantage of single camera stereo panoramic systems is that since there are two 
15 images of the "same" scene, the pixels assigned to each image is half that for non stereo 
panoramic imaging and the two images do not share an equal number of pixels in constant 
gain schemes. Actually, the panoramic stereo double mirror method typically causes the view 
of a scene in one radial direction to be compressed into around 1/4 the field of view of the 
camera. 

20 

A method to achieve panoramic stereo with less image compression is to use two cameras and 
two single curved mirror surfaces back to back, as shown in Fig. 9. This method compresses 
the imaged scene into 1/2 the field of view of the camera, and indeed each image has an equal 
share of the total number of pixels available. There are, however, possible alignment 
25 problems with this system as with any stereo system using two cameras to capture two views 
of a scene. 

An advantage of the scheme proposed in Fig. 9 is that the stereo cone can be symmetric about 
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the horizon using two cameras with equal fields of view and the maximum and minimum 
angles of elevation reflected by the two mirrors being equal. The angle covered by the stereo 
cone in this case is 2$ - n. Fig.9 shows the general case where the maximum and minimum 
angles of elevations viewed by each camera need not be equal. The range of elevations must 
5 still be equal for the fields of view to be aligned. 

The number of free parameters to be specified are reduced here as the minimum angle of 
elevation (4> ) and from one mirror must be parallel to the maximum angle of elevation ) 
from the other mirror. This is to ensure that the fields of view are parallel. So, with reference 
10 to Fig. 9 

4> = it - cj> (26) 
In the scheme of Fig. 9, the mirror families can be either constant gain or resolution invariant. 

15 

2.1 The Use of Double Mirrors in a Back to Back Design 

Fig. 10 shows a back to back design incorporating double mirrors. Although the figure shows 
20 constant gain mirrors, the double mirror can also have a variable gain. The advantage to this 
system is that the stereo cone from the back to back configuration combines with the stereo 
cones from the double mirror configuration to increase the total area imaged in stereo. In this 
configuration, the fields of view of each double mirror pair need not be aligned as in previous 
examples. For symmetry about the horizon ^ = ^ , ^ = ^ , 
25 <$T 3 = 4>j and fy 4 — <j) 2 • The constraints 
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1>~ 3 = K ~ <j>2 

<K = * - 4>i 

align the three stereo cones. 

It is also possible to increase the total stereo cone further by allowing the mirror pairs to have 
5 different gains. 

3 Forward Looking Mirror Design 

An example of a forward looking mirror design is shown in Fig. 11. For many applications, 
10 it is desirable to have a panoramic camera looking out from, say, a hemisphere, somewhat as 
an eye of a bird, or perhaps two such on either side of a "nose cone" . There are aerodynamic 
considerations or other protrusion considerations which motivate such a "forward looking" 
system. This configuration is termed forward looking because the camera faces towards the 
scene. Either a constant or variable gain mirror (double or single) could be used for the 
15 curved mirror in the system. The planar mirror is an annulus or circle interposed such that 
all rays reflected from the curved mirror are reflected into camera o positioned behind the 
curved mirror. The dotted lines in Fig. 11 show where the reflected rays would converge if 
the planar rnirror was removed and the dotted camera shows the camera o' for an equivalent 
system without the planar mirror. 

20 

In order for the rays reflected by the planar mirror to converge at the new camera position, 
the planar mirror must be the perpendicular bisector of the line joining the old and new 
camera locations. Hence the distance between the camera locations is 2D where D is defined 
in Fig. 11 as the distance from either camera to the planar mirror. The introduction of the 
25 planar mirror into the system does increase the possibility of alignment difficulties as the 
planar mirror must be perpendicular to the camera axis and also be positioned so as to reflect 
all rays from the curved mirror into the camera without occluding the view of the curved 
mirror. 
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The maximum value for D,D , is when the maximum beam reflected from the mirror system 
(the B beam reflected at point b on the planar mirror) into camera o grazes the curved profile 
at c. In this case 



rcos(9)[tan(e) +tan(6)] 

D — [27) 

2 tan (6) 



D , defines the niinimum height for the rnirror system, H . In practice, the value for D needs 
to be slightly smaller to avoid occlusion, leading to a larger mirror system height. The general 
equation for the height of the mirror system is 

10 

H = r cos (0) - D (28) 



It should also be noted that 6 must be greater than zero for camera o to be located behind 
the curved mirror. Also, § s 0 if the minimum elevation ray <j> is not to be occluded by 
the planar mirror. 

15 

Fig. 12 shows a design that incorporates the ideas of Sections 2 and 3. It consists of two 
forward looking systems back to back, giving a design reminiscent of a eye mounted on a 
stalk, such as a crab's eye. The "stalk" for this system would be hidden from view by the 
lower planar rnirror. In this arrangement portions are provided in the curved mirror to 
20 provide for reflection of rays from the curved surface to the camera by the plane mirrors. 
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The foregoing describes only some aspects of the present invention and modifications can be 
made without departing from the scope of the invention. 



5 
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CLAIMS: 



1 . A panoramic imaging system including an imaging device having an image plane and 
a first field of view, a first reflective surface having at least one circularly symmetric portion 
5 convex in a radial direction disposed in said first field of view to provide an expanded 
panoramic second field of view, the profile of the or each convex portion providing a varying 
gain between the fields of view in the radial direction to limit variation in the solid angle of 
view across the image plane of the imaging device. 

10 2. A panoramic imaging system as claimed in claim 1 wherein the profile of the or each 
convex portion provides a substantially uniform solid angle of view across the image plane. 

3. A panoramic imaging system as claimed in claim 1 or claim 2 wherein the profile of 
the or each convex portion at least approximates a profile defined in polar co-ordinates by the 
15 equation: 




where r is the radial distance from the reflective surface to the imaging device 
0 is the angle from the optical axis of the imaging device 
a (8) is the- mirror gain given by 



20 



a (0) = B a [tan (0) + tan 3 (0)] 



5 - 



2(4) - 4>) 



tan 2 



i 2 (6) - tan 2 (0) 



cj> and <J> are the maximum and minimum elevations viewed 13 and 6 are the maximum and 
minimum radial angles imaged. 
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4. A panoramic imaging system as claimed in claim 3 wherein the profile of the or each 
convex portion includes by a series spaced apart points defined by determining distance r for 
selected values of angle 6. 

5 5. A panoramic imaging system as claimed in claim 4 wherein the selected values of 6 
are separated by about 1/5°. 

6. A panoramic imaging system as claimed in any one of claims 1 to 5 including a first 
reflector surface having at least two of said convex portions arranged to respectively provide 

10 at least partially overlapping panoramic second fields of view for range determination. 

7. A panoramic imaging system as claimed in claim 6 wherein said panoramic second 
fields of view are substantially co-incident. 

15 8. A panoramic imaging system as claimed in claim 7 wherein said at least two convex 
portions form a continuous reflective surface. 

9. A panoramic imaging system as claimed in any one of claims 1 to 8 including two of 
said first reflective surfaces each having an associated image plane with corresponding first 

20 fields of view, and at least one convex portion of each first reflective surface providing 
respective panoramic second fields of view, said first reflective surface being arranged back 
to back such that said reflective second fields of view at least partially overlap. 

10. A panoramic imaging system as claimed in any one of claims 1 to 9 further including 
25 a second reflective surface interposed between the image plane and said second reflective 

surface. 



11. A panoramic imaging system as claimed in claim 10 wherein the imaging device is 
positioned behind the second reflective surface. 

30 



WO 00/42470 



PCT/AUOO/00022 



-23- 



12. A panoramic imaging system as claimed in claim 11 wherein an aperture is provided 
in said first reflective surface to provide said first field of view from the imaging device. 

5 13. A panoramic imaging system as claimed in any of claims 10 to 12 wherein said 
second reflective surface is substantially planar. 

14. A reflective surface for use in a panoramic imaging system including an imaging 
device having an imaging plane and a first field of view, said reflective surface having at least 
1 0 one circularly symmetric portion convex in a radial direction with a profile providing varying 
gain in the radial direction between an expanded panoramic second field of view provided by 
the reflective surface and the first field of view to limit variation in the solid angle of view 
across the image plane of the imaging device. 

15 15. A reflective surface as claimed in claim 14 wherein the profile of the or each convex 
portion provides a substantially uniform solid angle of view across the image plane. 

16. A reflective surface as claimed in claim 14 or claim 18 wherein the profile of the or 
each convex portion at least approximates a profile defined in polar co-ordinates by the 
20 equation: 




where r is the radial distance from the reflective surface to the imaging device 
6 is the angle from the optical axis of the imaging device 
a (0) is the mirror gain given by 



25 



a (6) = B a [tan (6) + tan 3 (6)] 



B n = 



2(4) - j>) 



tan 2 (6) - tan 2 (6_) 
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$ and 4» are the maximum and minimum elevations viewed 0" and© are the maximum and 
minimum radial angles imaged. 

17. A reflective surface as claimed in claim 16 wherein the profile of the or each convex 
5 portion includes by a series spaced apart points defined by determining distance r for selected 

values of angle 6. 

18. A reflective surface as claimed in claim 17 wherein the selected values of 0 are 
separated by about 1/5°. 

10 

19. A reflective surface as claimed in any one of claims 14 to 18 including a first reflector 
surface, having at least two of said convex portions arranged to respectively provide at least 
partially overlapping panoramic second fields of view for range determination. 

15 20. A reflective surface as claimed in claim 19 wherein said panoramic second fields of 
view are substantially co-incident. 

21 . A reflective surface as claimed in claim 20 wherein said at least two convex portions 
form a continuous reflective surface. 
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